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DETAILED ACTION 
L This action is responsive to communications: Amendment, filed on 7/30/04. 

2. Claims 1-29 are pending in this appHcation. Claims 1, 12, 15, and 21 are independent 
claims. In the Amendment, filed on 7/30/04, claims 1-7, 9-16, 18-21, and 23-29 were amended. 

Claim Rejections - 35 USC §102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 35 1 (a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

3. Claim 1-10, 12-19, 21, and 23-27, and 29 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Liou et al. (US 6,278,446). 

As per claim 12, Liou et al. teaches a video data structure tangibly embodied in a 
computer readable medium for a video browsing system comprising: 

a visualization description scheme (DS) which includes a highlight view DS for display 
for displaying event data by a highlight, and a key frame view DS for displaying event data as 
key frames, wherein the highlight view DS is organized into multiple levels which enables a 
display of multi-levels of highlight data, and wherein the key frame view DS is organized into 
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multiple levels which enables a display of multi- levels of summarized data (col. 11, lines 62-68; 
col. 12, lines 1-10) 

a syntactic structure DS which includes information for displaying actual video segments 
of a video (col. 15 lines 32-45); and 

a semantic structure DS which includes additional information describing the video, 
wherein the semantic description includes information indicative of a storyline relation between 
objects in the video (col. 14, lines 1-26). 

As per claim 13, Liou et al. teaches a video data structure of claim 12, wherein the 
syntactic structure DS is organized into a segment IDS including actual video segment data and a 
time DS including corresponding temporal positions of each actual video segment data within 
the video data (col. 11, lines 12-34). 

As per claim 14, Liou et al. teaches a video data structure of claim 13, wherein the 
semantic structure DS is organized into sub-level structures comprising: 

an event DS which includes event information, wherein said event DS is organized into 
sub-level structures of a reference to segment which includes reference information necessary for 
displaying a video segment of a video corresponding to events selected by a user and an 
annotation DS which includes information which connects said selected events with actual 
positions of said selected events within the video data and information explaining said selected 
events (fig 12, col. 11 lines 63-68, col. 12, Hnes 1-10); 

an object DS which includes object information (col. 12 , lines 1-20, fig. 12, the text next 
the images); and 
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an event/relation graph DS which includes information on at least one of constant story 
line relations between the objects, variable storyline relations between the objects, or story line 
relations between the objects and events, wherein said event/object relation graph DS is 
organized into an entity story line relation with sub-level structures of a Reference to Object 
which connects objects having either a constant story line relation or variable storyline relation, a 
Reference to Event which connects events which are significant to a story line relation between 
each connected object, and a story line relation which includes information on a nature and title 
of story line relation between each connected object (fig 12, col. 1 1 lines 63-68, col. 12, lines 1- 
10). 

As per claim 1, it is rejected with the same rationale as claim 12. (see rejection above) 
As per claim 2, Liou et al. teaches a system of claim 1, wherein the main screen displays 
said video segments by a summary data of said video segments, (fig. 12 items 34, 36, 38, and 40) 

As per claim 3, Liou et al. teaches a system of claim 1, wherein a user selects two 
characters through the character screen to display the video segments corresponding to 
significant events showing a the storyline relation between the two displayed characters (fig 12, 
col. 11 lines 63-68, col. 12, lines 1-10; fig 14. col. 13, lines 53-68, col 14, lines 1-6; It is 
inherent that two sub-branch of the plotline can be selected, and their plotline relationship to 
each other is displayed by the main plotline). 

As per claim 4, Liou teaches (Currently Amended) A system of claim 1, further 
comprises a character relation- variable event screen which displays connections between 
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variable story line relations and significant events (fig 12, col. 11 lines 63-68, col. 12, lines 1- 
10). 

As per claim 5, Liou teaches (Currently Amended) A system of claim 4, wherein the 
character relation- variable event screen displays either one or both the variable story line 
relations and events by key frames (fig 12, col. 1 1 lines 63-68, col. 12, lines 1-10). 

As per claim 6, Liou teaches A system of claim 1, fiirther comprising: 

a relation screen which displays constant story line relations and variable story line 
relations between a character selected from the character screen and related characters, wherein 
said constant story line relations and variable to line relations are displayed in a tree structure (fig 
12, col. 11 lines 63-68, col. 12, lines 1-10); and 

a main scene screen which displays significant events corresponding to one of 
either a constant story line relation or variable story line relation selected from the relation screen 
(fig 12, col. 11 lines 63-68, col. 12, lines 1-10). 

As per claim 7, Liou teaches a system of claim 6, wherein the constant sto Hne relation is 
displayed on a top level of said tree structure and variable sto line relations are displayed on 
lower levels of said tree structure (fig 12, col 11 lines 63-68, col. 12, lines 1-10). 

As per claim 8, Liou teaches a system of claim 6, wherein the main scene screen displays 
the significant events by key frames (fig 12, col. 11 lines 63-68, col. 12, lines 1-10). 

As per claim 9, Liou teaches a system of claim 6, fixrther comprising a selection screen 
wherein the main scene screen displays either one or both main significant events or secondary 
significant events corresponding to said storyline relation selected from the relation screen. 
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according to a user selection through said selection screen (fig 12, col. 1 1 lines 63-68, col 12, 
lines 1-10). 

As per claim 10, Liou teaches a system of claim 6, further comprising a selection screen, 
wherein the relation screen displays constant storyline relations and variable line relations 
corresponding to a story line relation type selected by a user through said selection screen (col. 5, 
lines 1-20). 

As per claim 15, Liou teaches a video browsing system comprising: 
a character screen which displays characters of a video ; 

a main screen which displays video segments corresponding to significant events 
showing a plotline relation between two displayed characters of said character screen according 
to a user selection, wherein said plotline relation may be constant or variable (fig 12, col. 1 1 lines 
63-68, col. 12, lines 1-10; fig 14. col. 13, lines 53-68, col. 14, lines 1-6; It is inherent that two 
sub-branch of the plotline can be selected, and their plotline relationship to each other is 
displayed by the main plotline); 

a relation screen which displays constant story plotline relations and variable plotline 
relations between a character selected from the character screen and related characters, wherein 
said constant plotline relations and variable plotline relations are displayed in a tree structure (fig 
12, col. 1 1 lines 63-68, col. 12, lines 1-10); and 

a main scene screen which displays significant events corresponding to one of either a 
constant plotline relation or variable plotline relation selected from the relation screen (fig 12, 
col. 11 lines 63-68, col. 12, lines 1-10). 



Application/Control Number: 09/645,613 Page 7 

Art Unit: 2174 

As per claim 16, Liou teaches a system of claim 15, wherein the constant plotline relation 
is displayed on a top level of said tree structure and variable plotline in relations are displayed on 
lower levels of said tree structure (fig 12, col 1 1 lines 63-68, col 12, lines 1-10). 

As per claim 17, Liou teaches a system of claim 15, wherein the character screen, the 
main screen, the relation screen, and the main scene screen are displayed using a video data 
structure comprising: 

a visualization DS which includes a highlight view DS for display for displaying event 
data by a highlight and a key fi^ame view DS for displaying event data as key fi*ames, wherein the 
highlight view DS is organized into multiple levels which enables a display of multi-levels of 
highlight data and wherein the key frame view DS is organized into multiple levels which 
enables a display of multi-levels of summarized data (fig 12, col 1 1 lines 63-68, col 12, lines 1- 
10); 

a syntactic structure DS which includes information for displaying actual video segments 
of a video; and a semantic structure DS which includes additional information describing a video 
(fig 12, col 11 lines 63-68, col 12, lines 1-10). 

As per claim 18, Liou teaches a system of claim 15, further comprising a selection screen 
wherein the main scene screen displays either one or both main significant events or secondary 
significant events corresponding to said plotline relation selected fi-om the relation screen, 
according to a user selection through said selection screen (fig 14, col 13, lines 53-68, col 14, 
lines 1-6). 
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As per claim 19, Liou teaches a system of claim 15, further comprising a selection screen 
wherein the relation screen displays constant plotline relations and variable plotline relations 
corresponding to a potline relation type selected by a user through said selection screen. 

As per claim 21, Liou teaches a video browsing system comprising: 

a character menu configured to displays characters that are part of a video; and 

a video display configured to display video segments from the video, wherein the video 
segments correspond to events showing a story line relationship between two displayed 
characters in said character menu (fig 14. col. 13, lines 53-68, col. 14, lines 1-6). 

As per claim 23, Liou teaches the system of claim 21, wherein a user selects two 
characters through the character menu to display the video segments corresponding to events 
showing the story line relationship between the two displayed characters (fig 14. col. 13, lines 
53-68, col. 14, lines 1-6). 

As per claim 24, Liou teaches the system of claim 21, further comprises a character 
relationship- variable event menu configured to display connections between variable story line 
relationships and events (fig 14. col. 13, lines 53-68, col. 14, lines 1-6). 

As per claim 25, Liou teaches the system of claim 24, wherein the character relationship- 
variable event menu configured to display either one or both the variable story line relationships 
and events by key frames (fig 14. col 13, lines 53-68, col. 14, lines 1-6). 

As per claim 26, Liou teaches the system of claim 21, further comprising: a relationship 
menu configured to display constant storyline relationships and variable storyline relationships 
between a character selected from the character menu and related characters, wherein said 
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constant story line relationships and variable story line relationships are displayed in a tree 
structure (fig 14. col 13, lines 53-68, col. 14, lines 1-6); and 

a main scene menu configured to display significant events corresponding at least one a 
constant story line relationship and a variable story line relationship selected from the 
relationship screen (fig 14. col. 13, lines 53-68, col. 14, lines 1-6). 

As per claim 27, Liou teaches the system of claim 26, v^herein the constant story line 
relationship is displayed on a top level old said tree structure and variable story line relationships 
are displayed on lower levels of said tree structure (fig 14. col. 13, lines 53-68, col. 14, lines 1- 
6). 

As per claim 29, Liou teaches the system of claim 21, wherein said story line relationship 
is at least one of a constant story line relationship and a variable storyline relationship (fig 14. 
col. 13, lines 53-68, col. 14, lines 1-6). 

Claim Rejections - 35 USC §103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 
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4. Claims 1 1, 20, 22, and 28 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Liou et al. (US 6,278,446) in view of Keith (US 6,629,097). 

As per claim 11, Liou teaches a system of claim 10. However he fails to teach wherein a 
store relation type may be one of a family relation, a business relation, or a social relation. 

Keith teaches where in store relation type may be on of a family relation, a business 
relation, or a social relation (col. 31, lines 40-68) 

It would have been obvious to an artisan at the time of the invention to include Keith's 
teaching with the system of Liou in order to allow stored video clip to be displayed in an entities 
and concepts relationship that visually reduce individual data items within such a cluster to one 
encapsulated object or icon representing all the individual data points within the cluster. . 

As per claim 20, it is of the same scope as claim 1 1 . (see rejection above) 

As per claim 28, it is of the same scope as claim 1 1 . (see rejection above) 

As per claim 22, Liou teaches the system of claim 21 . However, he fails to teach wherein 
the characters are represented by actors in the video. 

Keith teaches the characters in the system are represented by actors. 

It would have been obvious to an artisan at the time of the invention to include Keith's 
teaching with the system of Liou in order to allow stored video clip to be displayed in an entities 
and concepts relationship that visually reduce individual data items within such a cluster to one 
encapsulated object or icon representing all the individual data points within the cluster. 

Response to argument 

Applicant's arguments with respect to claims 1-29 have been considered but are deemed 
to be moot in view of the new grounds of rejection. 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Peng Ke whose telephone number is (571) 272-4062. The 
examiner can normally be reached on M-Th and Alternate Fridays 8:30-5:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kristine L Kincaid can be reached on (571) 272-4063. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or PubHc PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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